The number of international benchmarking competitions is steadily increasing in various fields of machine learning (ML) research and practice. So far, however, little is known about the common practice as well as bottlenecks faced by the community in tackling the research questions posed. To shed light on the status quo of algorithm development in the specific field of biomedical imaging analysis, we designed an international survey that was issued to all participants of challenges conducted in conjunction with the IEEE ISBI 2021 and MICCAI 2021 conferences (80 competitions in total). The survey covered participants' expertise and working environments, their chosen strategies, as well as algorithm characteristics. A median of 72% challenge participants took part in the survey. According to our results, knowledge exchange was the primary incentive (70%) for participation, while the reception of prize money played only a minor role (16%). While a median of 80 working hours was spent on method development, a large portion of participants stated that they did not have enough time for method development (32%). 25% perceived the infrastructure to be a bottleneck. Overall, 94% of all solutions were deep learning-based. Of these, 84% were based on standard architectures. 43% of the respondents reported that the data samples (e.g., images) were too large to be processed at once. This was most commonly addressed by patch-based training (69%), downsampling (37%), and solving 3D analysis tasks as a series of 2D tasks. K-fold cross-validation on the training set was performed by only 37% of the participants and only 50% of the participants performed ensembling based on multiple identical models (61%) or heterogeneous models (39%). 48% of the respondents applied postprocessing steps.

Extensive empirical evidence demonstrates that conditional generative models are easier to train and perform better than unconditional ones by exploiting the labels of data. So do score-based diffusion models. In this paper, we analyze the phenomenon formally and identify that the key of conditional learning is to partition the data properly. Inspired by the analyses, we propose self-conditioned diffusion models (SCDM), which is trained conditioned on indices clustered by the k-means algorithm on the features extracted by a model pre-trained in a self-supervised manner. SCDM significantly improves the unconditional model across various datasets and achieves a record-breaking FID of 3.94 on ImageNet 64x64 without labels. Besides, SCDM achieves a slightly better FID than the corresponding conditional model on CIFAR10.

最近，已经提出了许多有效的变压器，以降低由软磁性注意引起的标准变压器的二次计算复杂性。但是，他们中的大多数只是用有效的注意机制交换SoftMax，而无需考虑定制的体系结构，特别是为了有效的关注。在本文中，我们认为手工制作的香草变压器体系结构可用于软马克斯的注意力可能不适合有效的变压器。为了解决这个问题，我们提出了一个新框架，通过神经体系结构搜索（NAS）技术找到有效变压器的最佳体系结构。提出的方法在流行的机器翻译和图像分类任务上进行了验证。我们观察到，与标准变压器相比，有效变压器的最佳体系结构的计算降低，但总体准确性较低。这表明SoftMax的注意力和有效的注意力具有自己的区别，但它们都无法同时平衡准确性和效率。这激发了我们混合两种注意力以减少性能失衡。除了现有NAS变压器方法中常用的搜索空间外，我们还提出了一个新的搜索空间，该空间允许NAS算法与架构一起自动搜索注意变体。 WMT'EN-DE和CIFAR-10上的广泛实验表明，我们的搜索架构与标准变压器保持了可比的精度，并具有明显提高的计算效率。

扩散概率模型（DPM）是一类强大的深层生成模型（DGM）。尽管取得了成功，但在整个时间段上的迭代生成过程效率要比其他DGMS（例如gans）效率要低得多。因此，时间步长上的生成性能至关重要，这受到DPMS中协方差设计的极大影响。在这项工作中，我们考虑对角和完整的协方差，以提高DPM的表现力。我们得出此类协方差的最佳结果，然后在DPM的平均值不完善时将其纠正。最佳和校正后的都可以分解为对噪声功能的条件期望的术语。在此基础上，我们建议通过学习这些条件期望来估计最佳协方差及其校正。我们的方法可以应用于离散时间和连续时间段的DPM。我们在实施计算效率方面考虑了对角协方差。为了进行有效的实际实施，我们采用参数共享方案和两阶段的培训过程。从经验上讲，我们的方法的表现优于可能性结果的各种协方差设计，并提高了样本质量，尤其是在少数时间段上。

基于梯度的残差训练的方法（RESNET）通常需要输入数据的前进通过，然后将误差梯度反向传播到更新模型参数，这变得耗费网络变得更深。为了破坏前向和向后模式的算法锁定和利用同步模块并行性，辅助变量方法最近吸引了很多兴趣，但遭受了重大的沟通开销和缺乏数据增强。在这项工作中，通过交易外部辅助变量的存储和重新计算，建立了一种用于跨多个计算设备训练现实Resnet的新颖联合学习框架。更具体地，每个独立处理器的输入数据是从其低容量辅助网络（AUXNET）生成的，这允许使用数据增强并实现前向解锁。然后并行地执行后向通过，每个丢失函数源自惩罚或增强拉格朗日（AL）方法。最后，采用所提出的AUXNET通过端到端培训过程重现更新的辅助变量。我们展示了我们在CIFAR-10，CIFAR-100和Imagenet数据集中展示了我们对RESNET和WIMERESNET的效果，实现了传统的层串行训练方法的加速，同时保持了可比的测试精度。

路径规划是自治车辆运动规划中的关键组成部分。路径指定车辆将旅行的几何形状，因此，对安全和舒适的车辆运动至关重要。对于城市驾驶场景，自治车辆需要能够在杂乱的环境中导航，例如，道路部分被侧面挡住的车辆/障碍物。如何生成运动学上可行和平滑的路径，可以避免复杂环境中的碰撞，使路径规划有挑战性的问题。在本文中，我们提出了一种新型二次编程方法，可以产生分辨率完全碰撞避免能力的最佳路径。

Learning effective motion features is an essential pursuit of video representation learning. This paper presents a simple yet effective sample construction strategy to boost the learning of motion features in video contrastive learning. The proposed method, dubbed Motion-focused Quadruple Construction (MoQuad), augments the instance discrimination by meticulously disturbing the appearance and motion of both the positive and negative samples to create a quadruple for each video instance, such that the model is encouraged to exploit motion information. Unlike recent approaches that create extra auxiliary tasks for learning motion features or apply explicit temporal modelling, our method keeps the simple and clean contrastive learning paradigm (i.e.,SimCLR) without multi-task learning or extra modelling. In addition, we design two extra training strategies by analyzing initial MoQuad experiments. By simply applying MoQuad to SimCLR, extensive experiments show that we achieve superior performance on downstream tasks compared to the state of the arts. Notably, on the UCF-101 action recognition task, we achieve 93.7% accuracy after pre-training the model on Kinetics-400 for only 200 epochs, surpassing various previous methods

Despite the surprising few-shot performance of in-context learning (ICL), it is still a common practice to randomly sample examples to serve as context. This paper advocates a new principle for ICL: self-adaptive in-context learning. The self-adaption mechanism is introduced to help each sample find an in-context example permutation (i.e., selection and ordering) that can derive the correct prediction, thus maximizing performance. To validate the effectiveness of self-adaptive ICL, we propose a general select-then-rank framework and instantiate it with new selection and ranking algorithms. Upon extensive evaluation on eight different NLP datasets, our self-adaptive ICL method achieves a 40% relative improvement over the common practice setting. Further analysis reveals the enormous potential of self-adaptive ICL that it might be able to close the gap between ICL and finetuning given more advanced algorithms. Our code is released to facilitate future research in this area: https://github.com/Shark-NLP/self-adaptive-ICL

In recent years, object detection has achieved a very large performance improvement, but the detection result of small objects is still not very satisfactory. This work proposes a strategy based on feature fusion and dilated convolution that employs dilated convolution to broaden the receptive field of feature maps at various scales in order to address this issue. On the one hand, it can improve the detection accuracy of larger objects. On the other hand, it provides more contextual information for small objects, which is beneficial to improving the detection accuracy of small objects. The shallow semantic information of small objects is obtained by filtering out the noise in the feature map, and the feature information of more small objects is preserved by using multi-scale fusion feature module and attention mechanism. The fusion of these shallow feature information and deep semantic information can generate richer feature maps for small object detection. Experiments show that this method can have higher accuracy than the traditional YOLOv3 network in the detection of small objects and occluded objects. In addition, we achieve 32.8\% Mean Average Precision on the detection of small objects on MS COCO2017 test set. For 640*640 input, this method has 88.76\% mAP on the PASCAL VOC2012 dataset.

The ability to associate touch with sight is essential for tasks that require physically interacting with objects in the world. We propose a dataset with paired visual and tactile data called Touch and Go, in which human data collectors probe objects in natural environments using tactile sensors, while simultaneously recording egocentric video. In contrast to previous efforts, which have largely been confined to lab settings or simulated environments, our dataset spans a large number of "in the wild" objects and scenes. To demonstrate our dataset's effectiveness, we successfully apply it to a variety of tasks: 1) self-supervised visuo-tactile feature learning, 2) tactile-driven image stylization, i.e., making the visual appearance of an object more consistent with a given tactile signal, and 3) predicting future frames of a tactile signal from visuo-tactile inputs.